AI-Powered Natural Language Programming Streamlines Coding Process

Whenever media layoffs are announced, internet trolls and tech bros often use the phrase “Learn to code” as a pejorative. However, with the emergence of code generating AIs, knowing a programming language like Python may soon become as useful as knowing a dead language like Sanskrit. These genAIs are already assisting professional software developers in coding more efficiently by taking on much of the programming grunt work. Therefore, the sentiment of “Learn to code” is not only useless but also becoming outdated.

How coding works

Two of the most widely used and written coding languages today are Java and Python. The former almost single-handedly revolutionized cross-platform functionality when it was released in the mid-1990s, and now powers “everything from smart cards to space vehicles” in 2020, according to Java Magazine — not to mention Wikipedia’s search function and all things Minecraft. The latter actually preceded Java by a few years and serves as the code base for many modern applications such as Dropbox, Spotify and Instagram.

They differ significantly in their functionality in that Java must be compiled (which means that its human-readable code must be translated into machine code that can be executed on a computer) before it can run, while Python is an interpreted language, meaning that its human code is converted to machine code. line by line during program execution, allowing it to run without first being compiled. The interpreted method makes it easier to write code for multiple platforms, while compiled code tends to focus on a specific type of processor. Regardless of how they work, the actual process of writing code is nearly identical between the two: someone has to sit down, open a text editor or Integrated Development Environment (IDE), and write all those lines of instructions. And until recently, that someone was typically a human.

The process of writing today’s “classical programming” is not so different from that of ENIAC, where a software engineer takes a problem, breaks it down into a set of subproblems, and writes code to solve each of those subproblems. in order, and then repeatedly debugging and recompiling the code until it executes. “Automatic programming” on the other hand removes the programmer at a certain distance. Instead of writing each line of code individually, the person creates a high-level abstraction of the task so that the computer then creates low-level code to address. This differs from “interactive” programming, which allows you to code a program while it is already running.

Today’s conversational AI coding systems, such as Github’s Copilot or OpenAI’s ChatGPT, further remove the programmer by hiding the coding process behind a veneer of natural language. The programmer tells the AI what they want to program and how, and the machine can automatically generate the necessary code.

Building tools to build tools, which allows you to build tools

One of the first of these new conversational coding AIs was Codex, developed by OpenAI and released in late 2021. OpenAI had already implemented GPT-3 (predecessor to GPT-3.5, which serves as BingChat’s public) at this point, a major language. a model that is remarkably adept at imitating human speech and writing after being trained on billions of words from the public web. The company then fine-tuned the model using more than 100 gigabytes of GitHub data to create Codex. It can generate code in 12 different languages and can translate existing programs between them.

Codex is adept at creating small, simple or repeatable assets such as “a big red button that vibrates the screen for a moment when clicked” or common actions such as email address verification in a Google Web Form. But no matter how prolific your prose is, you won’t use it for complex projects like coding a server-side load balancer—it’s just too complicated a question.

Google’s DeepMind developed AlphaCode specifically to address such challenges. Like Codex, AlphaCode was first trained on several gigabytes of GitHub code repositories, but then fed thousands of coding challenges from online programming competitions, such as finding out how many binary strings of a given length do not contain consecutive zeros.

To do this, AlphaCode generates up to a million code candidates and then rejects all but the top 1 percent of its test cases to pass. The system then groups the rest of the programs based on the similarity of their results and tests them one after the other until it finds a candidate that solves a given problem. According to a 2022 study published in the journal Science, AlphaCode was able to correctly answer these challenge questions 34 percent of the time (compared to Codex’s single-digit success with the same benchmarks, that’s not bad). DeepMind even entered AlphaCode, an online programming competition of 5,000 competitors, where it beat nearly 46 percent of human competitors.

Now even AI has notes

Just as GPT-3.5 serves as the base model for ChatGPT, Codex serves as the foundation for GitHub’s Copilot AI. Trained on billions of lines of code compiled from the public web, Copilot provides cloud-based AI-assisted coding auto-completion capabilities with an on-demand plug-in for Visual Studio Code, Visual Studio, Neovim, and JetBrains’ integrated development environments (IDEs).

Originally released as a developer preview in June 2021, Copilot was among the first coding-capable AIs to hit the market. More than a million developers have taken advantage of the system in the two years since, GitHub product director Ryan J Salva told ReturnByte in a recent interview. With Copilot, users can create executable code from natural language text inputs and automatically complete frequently repeated code sections and programming functions.

Salva notes that before Copilot was released, users accepted GitHub’s previous machine-generated coding suggestions only 14 to 17 percent of the time, “which is good. It means it helped developers move forward.” In the two years since Copilot’s debut, that number has grown to 35 percent, “and that’s just under half of the code being written [on GitHub] — 46 percent of AI, to be exact.”

“[It’s] not just about the percentage of code written,” Salva clarified. “It’s really about productivity, focus and satisfaction for creative developers.”

As with the output of natural language generators like ChatGPT, the code coming out of Copilot is largely readable, but like any large language model trained on the open internet, GitHub made sure to include additional safeguards to prevent the system from accidentally producing exploitable code.

“During the time that the model produces a proposal and when that proposal is presented to the developer,” Salva said, “we perform a runtime … code quality analysis for the developer, looking for common errors or vulnerabilities in the code, such as cross-site scripting or path injection.”

The purpose of this auditing phase is to improve the quality of the recommended code over time, rather than to monitor or control what the code might be used for. Copilot can help developers create malicious code, but the system does not prevent it. “We’ve taken the position that Copilot is a tool that helps developers produce code,” Salva said, referring to numerous White Hat applications for such a system. “Putting a tool like Copilot in their hands … makes them even more capable security researchers,” he continued.

As technology develops, Salva sees generative artificial intelligence coding expanding far beyond the current technological boundaries. This includes a “big bet” on conversational AI. “We’re also seeing AI-assisted development really seep into other parts of the software development lifecycle,” he said, such as using AI to fix CI/CD build errors, fix security vulnerabilities, or verify AI as written by a human. code.

“Just as we use compilers to produce machine-level code today, I think they will eventually reach a new level of abstraction with artificial intelligence that allows developers to express themselves in a different language,” Salva said. “Maybe it’s a natural language like English, French or Korean. And then it ‘combines’ into something that machines can understand,” freeing engineers and developers to focus on the overall growth of the project rather than the nuts and bolts of building it.

From coders to gabbers

Since human decision making is still firmly wedged into the AI programming loop, at least for now we have little to fear from the existence of software writing software. As Salva pointed out, computers already do this to some extent when compiling the code, and the digital gray goths haven’t yet taken over because of it. Instead, the most immediate challenges of programming AI mirror the general challenges of generative AI: inherent biases that distort training data, model outputs that infringe copyrights, and concerns about the privacy of surrounding users’ data when it comes to training large language models.

GitHub is far from alone in its efforts to build an AI programming companion. OpenAI’s ChatGPT can generate code – as well as countless indie versions built on top of the GPT platform. So is Amazon’s AWS CodeWhisperer system, which provides much of the same autocomplete functionality as Copilot, but is optimized for use within the AWS framework. After several requests from users, Google also included code generation and debugging features in Bard last April ahead of ecosystem-wide AI at I/O 2023 and the launch of Codey, Alphabet’s answer to Copilot. We can’t yet be sure what generative coding systems will eventually become, or how it might impact the tech industry—we can look at the earliest iterations of a transformative democratizing technology, or it might be Clippy for a new generation.